Navigation using augmented reality

ABSTRACT

Embodiments disclose a computer-implemented method for guiding a user along a route, the method including receiving route information from a route server, where the route information includes geographical positions of roads and designated lanes of travel for each of a plurality of segments along the route, and receiving a series of images captured by the camera along the route. The method includes analyzing the series of images to determine a horizon and one or more available lanes of travel along the route, identifying one or more designated lanes from the one or more available lanes based on the analyzing of the series of images, and displaying the series of images along with a navigational layer superimposed over the series of images to the user, the navigational layer comprising a blocked region covering areas of the series of images outside of the one or more designated lanes and below the horizon.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/251,771, filed Jan. 18, 2019, entitled “Navigation Using AugmentedReality,” which claims priority under 35 U.S.C. § 119(e) to U.S.provisional application No. 62/619,597, filed Jan. 19, 2018, entitled“Navigation Using Augmented Reality,” the contents of each of which areherein incorporated by reference.

BACKGROUND

Modern mobile devices (e.g., smartphones) can run applications that helpnavigate the user from one location to another. These navigationalapplications can utilize GPS coordinates to determine the location of amobile device and plot an intended travel route that can be used toguide the user from the location to a destination location. As the usertraverses the intended travel route, such applications provide a topdown view of a grid of streets and directional data to instruct the userwhere to turn along the intended travel route. The directional data caninclude symbols, such as arrows, for guiding the user along route. Thetime and effort required by the user to interpret these symbols caninhibit the ease at which the intended route is traversed and can leadto misdirection.

BRIEF SUMMARY

Embodiments can provide for improved devices, interfaces, and methodsfor navigation along a route between two locations, e.g., by providinglane guidance along the route. For instance, some embodiments canaugment a user's perceived reality by capturing images in front of theuser (e.g., in front of a vehicle) and superimposing a navigationallayer over a display of those captured images in real time. Bysuperimposing the navigational layer over the captured images, the usercan clearly and easily understand which lane to be positioned in withouthaving to decipher how general symbols correlate with the real world.

In some embodiments, a computer-implemented method for guiding a useralong a route between at least two locations includes, at a mobiledevice that has a display and a camera positioned to take images infront of a vehicle, receiving route information from a route server,where the route information includes geographical positions of roads anddesignated lanes of travel for each of a plurality of segments along theroute, and receiving a series of images captured by the camera along theroute. The method includes analyzing the series of images to determine ahorizon and one or more available lanes of travel along the route,identifying one or more designated lanes from the one or more availablelanes based on the analyzing of the series of images, and displaying theseries of images along with a navigational layer superimposed over theseries of images to the user, the navigational layer comprising ablocked region covering areas of the series of images outside of the oneor more designated lanes and below the horizon.

Other embodiments are directed to systems, portable consumer devices,and computer readable media associated with methods described herein.

A better understanding of the nature and advantages of embodiments ofthe present invention may be gained with reference to the followingdetailed description and the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B are snapshots illustrating exemplary interfaces providedby conventional devices for navigation.

FIG. 2 is a snapshot illustrating an exemplary navigational interface ofan augmented reality device, according to some embodiments of thepresent disclosure.

FIG. 3 is a snapshot illustrating an exemplary implementation of anaugmented reality device, according to some embodiments of the presentdisclosure.

FIG. 4 is a block diagram illustrating an exemplary augmented realitysystem that can be implemented in an augmented reality device to enablethe features of navigation and lane guidance using augmented reality,according to some embodiments of the present disclosure.

FIG. 5A is a simplified top-down diagram of a vehicle traveling along asegment of an exemplary route, according to some embodiments of thepresent disclosure.

FIGS. 5B and 5C are snapshots illustrating a navigational layersuperimposed over captured images for an exemplary case where only onelane in a multi-lane segment of road is a designated lane, according tosome embodiments of the present disclosure.

FIG. 6A is a simplified top-down diagram of a vehicle traveling along asegment that is a continuation of the exemplary route discussed in FIG.5A, but is now approaching an upcoming right turn, according to someembodiments of the present disclosure.

FIGS. 6B and 6C are snapshots illustrating a navigational layersuperimposed over captured images for an exemplary case where thedesignated lane has shifted to an adjacent lane, according to someembodiments of the present disclosure.

FIG. 7A is a simplified top-down diagram of vehicle traveling along asegment that is a continuation of the exemplary route discussed in FIG.6A, but is now encountering the upcoming right turn, according to someembodiments of the present disclosure.

FIG. 7B is a snapshot illustrating a navigational layer superimposedover a captured image for the exemplary case of FIG. 7A where thesegment of the designated lane is a turn, according to some embodimentsof the present disclosure.

FIG. 8A is a simplified top-down diagram of a vehicle traveling along asegment of an exemplary route having an unusual turn, according to someembodiments of the present disclosure.

FIG. 8B is a snapshot illustrating an exemplary navigation layerincluding a pop-up window after a threshold distance has been crossed,according to some embodiments of the present disclosure.

FIG. 8C is a snapshot illustrating an exemplary navigation layerincluding a pop-up window as the vehicle in FIG. 8B encounters theu-turn, according to some embodiments of the present disclosure.

FIG. 9 is a block diagram of a method for performing navigation and laneguidance, according to some embodiments of the present disclosure.

FIG. 10 is a block diagram of an example device for performingnavigation and lane guidance, according to some embodiments of thepresent disclosure.

DETAILED DESCRIPTION

An augmented reality device, according to some embodiments of thepresent disclosure, can enhance and improve the ease at which a devicecan convey lane guidance information to a user that is traveling along aroute. The device can capture a series of images (e.g., video footage)in front of the augmented reality device that closely represents what isactually perceived by the user, and display the captured series ofimages in real time, e.g., on a screen of the device or as a projection.Concurrently, the device can superimpose a navigational layer over thedisplayed captured images to guide the user into one or more lanes thatwould best position the user to proceed along the route.

To enable this functionality, the augmented reality device can beconfigured to communicate with one or more global positioning system(GPS) satellites and one or more route servers. The GPS satellites canutilize triangulation to provide the location of the device, and theroute server can utilize one or more databases to provide routeinformation regarding the route of travel between an origin location anda destination location (or to each intervening location between theorigin and destination locations). The route information can include,but is not limited to, road information (e.g., identification of roadsalong the route, what segments of those road will be traveled, thegeographical position of those roads, the geographical position of laneswithin each road, and in what order will the roads be traveled), turninformation (e.g., what turns are needed along the route and in whatorder), lane information (e.g., number and position of lanes for eachroad segment), and distance information (e.g., distance to travel oneach segment of road). In some embodiments, the augmented reality devicecan also be configured to communicate with any other positioningsystems, such as, but not limited to, wireless fidelity (WiFi)-basedpositioning systems, cellular based positioning systems, satellite-basedpositioning systems, and any other global navigation satellite systems(GNSS).

With this information, the augmented reality device can analyze thevideo footage to determine the number of available lanes. The device canthen correlate the video footage with the location of the device, whichcan be taken as the location of the vehicle, and identify one or moredesignated lanes from the one or more available lanes in which thevehicle should be traveling to stay in the route. The device can displaythe series of images along with a navigational layer over the series ofimages to guide the user into the correct lane, or a plurality ofcorrect lanes, if applicable.

In some embodiments, the navigational layer can block out regions (e.g.,lanes, curbs, sidewalks) of the captured images where the vehicle is notintended to travel, and leave unblocked regions (e.g., lanes) where thevehicle is intended to travel. That way, the user can clearly understandwhich lane he or she should be in. In additional embodiments, thenavigational layer can include an indicator positioned over theunblocked region. The indicator can be a single guidance arrow or aseries of guidance arrows that point toward the direction in which theuser should travel. For instance, the indicator can be a straight arrowthat points diagonally to the right, indicating that the user shouldswitch lanes to the right. In another instance, the indictor can be aseries of multiple diagonal arrows that blink in a sequential orderindicating the same. The indicator can blink to convey a degree ofurgency at which the user should perform the indicated task.Additionally, in some embodiments, a pop up window can appear in thenavigational layer to provide more information to the user about theupcoming turn. For example, the pop up window can be a birds eye view ofa turn that the user is about to take.

That way, the user can better understand what is about to happen.

Superimposing a navigational layer over video footage of what the usercan see helps clearly communicate navigational and lane guidanceinformation to the user. The augmented reality device can providesignificant improvements in navigation and lane guidance overconventional devices that merely provide symbols to guide a user along aroute. Devices that provide navigation using symbols require the user todecipher the symbols in real time and immediately apply them to what theuser is perceiving, which may not always be easy to do.

I. Navigation Using Symbols

Traveling from an origin location to a destination location is a trivialconcept; but in the real world, traveling between these two locationscan be extremely complex because it can involve traversing manydifferent roads that extend in varying directions, each potentially witha different number of lanes. This can get exponentially more difficultif there are intervening destinations (e.g., pit stops) between the twolocations. Naturally, the large network of roads lends itself to providemore than one way to get from the origin location to the destinationlocation. Some routes may be more direct, while others may take lesstime to travel. In an effort to determine the shortest route (orquickest route), modern mobile devices can be equipped with applicationsthat direct a user, in a step-by-step fashion, along a route from theorigin location to the destination location.

Often times, these instructions are communicated to the user withgeneral symbols and icons or with an arrow along a highlighted route,requiring the user to interpret these symbols and apply them to what theuser is perceiving in the real world. The time and effort required tointerpret these symbols can inhibit the ease at which the intended routeis traversed and can lead to misdirection.

FIGS. 1A and 1B illustrate exemplary interfaces provided by conventionaldevices for navigation. As shown in FIG. 1A, interface 100 includes abirds eye view of a portion of an application-generated map surroundinga user, which is represented by an arrow head 102. Arrow head 102 isshown traversing along a route 104 illustrated as a blue line on a road106 illustrated as a yellow stripe. Upcoming turns are shown as a symbol108 on the top left side of interface 100, and current road/streetinformation is shown on the top right side of the interface 100. Withthis interface, the user has to look away from the road to look at thedevice, interpret the map surrounding the user, interpret symbol 108,make sure the blue line has not moved, and determine whether the user ison the correct road, all of which require diversion of attention fromthe road and expense of mental capacity that distracts from the user'sability to drive the vehicle.

To minimize the amount of distraction these navigation devices cancause, some conventional navigation devices project instructions andsymbols onto a head-up display. A head up-display is a transparentdisplay created by reflecting an image off of a transparent panel thatpresents data without requiring a user to look away from his or herusual viewpoint. FIG. 1B illustrates an exemplary head-up displayinterface 112 provided by a conventional navigation device 114. Contentsdisplayed by device 114 can reflect off a vehicle windshield to presenthead-up display interface 112. The contents of interface 112 includeupcoming route information in the form of symbol 116 and additionalroute information, such as distance 118 to the next turn and/or thespeed limit 120. With such interfaces, the user does not have to lookaway from the road, but the user still has to interpret symbol 116 andtranslate that into what the user is perceiving in the real world. Insome instances, translating the symbol to apply to the real world can bementally taxing and can take time to decipher, which can hinder theusability of the navigational device.

II. Navigation and Lane Guidance Using Augmented Reality

Rather than using mere symbols or application-generated maps with routelines and arrows for navigation, embodiments of the present disclosureutilize augmented reality to perform navigation and lane guidance.Augmented reality describes a technology in which a live view of thereal world is supplemented with computer-generated data, such as text,graphics, or audio. In other words, the real world as seen by anaugmented reality device is enhanced with additional features. With theuse of augmented reality, the real world may become interactive andinformative. For example, information about a route may be overlaid ontoa real-world scene (perceived by the user as the user is traveling alonga segment of the route) to provide the user with more information aboutthe path of travel along the segment.

FIG. 2 illustrates an exemplary navigational interface 202 provided byan augmented reality device 200, according to some embodiments of thepresent disclosure. Augmented reality device 200 can be any suitabledevice configured to capture or receive a series of images (e.g., avideo), perform calculations on the captured images, and display theseries or images to a user on a display screen 203 while concurrentlysuperimposing a navigational layer over the series of images. In someembodiments, augmented reality device 200 is a smart device, such as asmart phone, or a wearable device, such as a pair of smart glasses.

In some embodiments, augmented reality device 200 can generate andsuperimpose a navigational layer over the series of image as it is beingdisplayed to the user in real time. For example, device 200 can captureimages of a scene 204 from a viewpoint similar to that of a user'sviewpoint as he or she is driving a vehicle. The captured images ofscene 204 can be displayed by the augmented reality device 200 as videofootage 206. As video footage 206 is displayed to the user, augmentedreality device 200 can superimpose a navigational layer over videofootage 206. In some embodiments, the navigational layer can becomputer-generated data that provides more information about thecaptured images than what is merely perceived from the viewpoint of theuser.

The navigational layer can include a blocked region 208 and an unblockedregion 210. Blocked region 208 can be a semi-opaque layer that covers aportion of video footage 206. The semi-opaque layer can have a manmade,artificial appearance so that a user does not mistake the semi-opaquelayer to be an actual, real-world obstacle. In some embodiments, blockedregion 208 can be any other pattern suitable for covering a portion ofvideo footage 206. For instance, blocked region 208 can be a hashedpattern, checkerboard pattern, chevron pattern, and the like to indicatethat the portion of video footage 206 is blocked. Blocked region 208 canalso be, in some embodiments, a monochromatic pattern that displays theportion of video footage 206 in a black-and-white color scheme toindicate that the portion of video footage 206 is blocked. Contentcovered by blocked region 208 can still be visible so that the user canstill see and be aware of what is covered by blocked region 208, asshown in FIG. 2. Blocked region 208 can represent areas that would leadthe user outside of the route, such as one or more lanes that end upcausing the user to deviate from the route, or be difficult to maketurns at certain decision points.

Unblocked region 210 can be a portion of the navigational layer thatdoes not block video footage 206, thereby allowing video footage 206 inthe unblocked region 210 to be clearly displayed. In some embodiments,unblocked region 210 can represent areas that lead the user along theintended route. By unblocking where the user needs to be, interface 202can clearly indicate where the user needs to be to proceed along theroute without requiring the user to decipher how a symbol translates towhat is perceived from his or her viewpoint.

In certain embodiments, blocked region 208 and unblocked region 210 canbe positioned below a horizon 212, as shown in FIG. 2. Horizon 212 cancorrelate with the actual horizon seen from the user's viewpoint andthus represent a horizontal division between the sky and the ground. Byproviding horizon 212, the navigational layer can clearly indicate thatthe blocked region 208 applies to the ground upon which the user istraveling and not the sky.

The combination of displaying blocked and unblocked regions enables thenavigational layer to clearly communicate in a straight forward mannerwhere the user should be to proceed along the route to get to thedestination location, and where the user should not be to avoiddeviating from the route. That way, the user will not have to deciphersymbols and correlate the symbol with what the user perceives.Accordingly, device 200 is an improved device over conventional devicesthat only provide symbols for the user to decipher.

In addition to blocked region 208 and unblocked region 210, thenavigational layer can also include additional features such as one ormore indicators and texts. For instance, the navigational layer caninclude an indicator 216 that visually communicates to the user theintended direction of travel. The navigational layer can also includetext 218, such as the name of the street, road, or highway along whichthe user is currently traveling. Furthermore, the navigational layer caninclude additional road information, e.g., speed limit information 220,to inform the user what the maximum speed of travel is for the currentroad. The navigational layer can also include more route information,such as upcoming turn information in the form of a pop-up window. All ofthese features will be discussed in detail further herein.

III. Implementation of an Augmented Reality Device

FIG. 3 illustrates an exemplary implementation 300 of an augmentedreality device 302, according to some embodiments of the presentdisclosure. Augmented reality device 302 can be used in a vehicle 304that is capable of being driven or otherwise controlled by a user. Insome embodiments, augmented reality device 302 can include a camera (notshown) for capturing images of front area 306, or can include hardwareand software for communicating with a separate camera positioned tocapture images of front area 306. If device 302 includes a camera, thendevice 302 can be positioned against a windshield or anywhere withinvehicle 304 so that device 302 can perceive a front area 306 of vehicle304. In some embodiments, device 302 can be integrated into vehicle 304.

To enable augmented reality device 302 to provide navigational and/orlane guidance functions, augmented reality device 302 can be configuredto communicate with external devices, such as one or more GPS satellites308 and a route server 310 through a wireless network 312. GPS satellite308 can provide GPS coordinates of device 302 that represent ageographical location of device 302. Route server 310 can be anapplication server that receives an origin location and a destinationlocation from device 302 (including any intervening locations betweenthe origin and destination location) and provides route information(e.g. in map form) for traveling between the two locations (as well asbetween any intervening locations along the route). Route informationcan be communicated to device 302 via wireless network 312, which can beany suitable network with which device 302 can interact to receive andsend information with route server 310, such as any suitable cellularnetwork (e.g., LTE, CDMA, GSM, and the like).

Augmented reality device 302 can be configured to receive location datafrom GPS satellite 308 and route information from route server 310,analyze captured images to determine available lanes in front of vehicle304, apply the route information and location data to the analyzedimages, and then superimpose a navigational layer over the capturedimages while the captured images are concurrently displayed to the user,thereby providing an augmented reality interface to a user, e.g.,interface 202 in FIG. 2. According to some embodiments of the presentdisclosure, augmented reality device 302 can include an augmentedreality system that enables device 302 to perform these functions.

FIG. 4 is a block diagram illustrating an exemplary augmented realitysystem 400 that can be implemented in an augmented reality device toenable the features of navigation and lane guidance using augmentedreality, according to some embodiments of the present disclosure.

Some blocks shown in FIG. 4 can be software modules stored in memory ofan augmented reality device, e.g., device 302 in FIG. 3, and accessibleby a processor (not shown) to execute the software module to perform aspecific function. Other blocks can represent data and/or signals thatare received by or sent from the software modules.

Augmented reality system 400 includes a navigation module 402 configuredto receive an input signal 404. Input signal 404 can be generated by aninput device, which can be a touch sensitive display, keyboard, and thelike. In some embodiments, input signal 404 can be a user inputindicating that the user desires to find a route between two locations.If known by the user, input signal 404 can also include addresses of theorigin location, designation location, or both. If the current locationof the device is unknown, then navigation module 402 can query andreceive GPS coordinates 406 from a GPS satellite. The received GPScoordinates 406 can then be used as the origin location for determiningthe route to the destination location.

Navigation module 402 can also query and receive route information 408from a route server through a communication network, as discussed hereinwith respect to FIG. 3. Route information 408 can include informationpertaining to the route between the origin location and the destinationlocation. For example, route information can include, but is not limitedto, road information (e.g., identification of roads along the route,what segments of those road will be traveled, the geographical positionof those roads, the geographical position of lanes within each road, andin what order will the roads be traveled), turn information (e.g., whatturns are needed along the route and in what order), lane information(e.g., number and position of lanes for each road segment), and distanceinformation (e.g., distance to travel on each segment of road).Depending on the user's route preference in terms of fastest route,shortest route, and the like, navigation module 402 can determine adesired route between the origin location and destination location.Navigation module 402 can also identify route information for thedesired route between the origin location and designation location. Oncethe route information is identified, navigation module 402 can comparethe vehicle's GPS coordinates 406 with the geographical position of theroad and the geographical position of the lanes within the road todetermine which lane the vehicle is currently traveling in.

Augmented reality system 400 can also include an image analysis module410. Image analysis module 410 can receive a series of captured images414 that may be captured by a camera. In some instances, captured images414 is real-time video footage of an area in front of a vehicle, e.g.,front area 306 discussed herein with respect to FIG. 3, while thevehicle is being driven by a user. Image analysis module 410 cananalyze, in real time, captured images 414 to identify and distinguishthe position of one or more objects present in captured images 414. Forinstance, image analysis module 410 can analyze captured images 414 toidentify one or more available lanes in the road on which the vehicle iscurrently traveling. In some instances, the available lanes can beidentified by the position of lane markers or dotted lane dividers onthe ground. Additionally, image analysis module 410 can determine thelocation of each dotted lane in the captured image. In some embodiments,image analysis module 410 can also determine the location of eachavailable lane and the position of the vehicle in relation to thoseavailable lanes. The available lanes can be the number of lanes in aroad on which a vehicle can travel. Image analysis module 410 can alsoidentify the curvature of the lane, the presence of road barriers, andany other physical object that the user may want to be aware of whiletraveling on the road.

According to some embodiments of the present disclosure, augmentedreality system 400 can include an augmented reality module 416.Augmented reality module 416 can receive the route information for thedesired route as well as the identity and position of objects analyzedfrom the captured images, and then analyze the route information and theidentified objects to determine how the route information correlateswith the identified objects. For example, augmented reality module 416can receive GPS coordinates of the current position of the vehicle androute information from navigation module 402, as well as theidentification and position of the available lanes as determined byimage analysis module 410. Augmented reality module 416 can thencorrelate the current position of the vehicle with where the vehicle isalong the route, and then determine which available lane from thecaptured images should be the designated lane, i.e., the lane in whichthe vehicle should be traveling to proceed along the route. If thevehicle is on a road with multiple lanes and is far away from the nextturn, then any of the available lanes can be a designated lane becausethe user just needs to be traveling forward. If, however, the vehicle ison a road with multiple available lanes but is approaching a turn, thenthe designated lane can be the far right lane (or far left lane) so thatit can be properly positioned to take the turn. Once the designated laneis determined, augmented reality module 416 can create a suitablenavigational layer that would indicate to a user to travel in thedesignated lane. For instance, augmented reality module 416 can create anavigational layer that unblocks a far right lane and blocks all otherlanes.

Once this correlation is made between the route information and theidentified objects, augmented reality module 416 can create a suitablenavigational layer that would indicate to a user to travel in thedesignated lane and output this navigational layer to display module418. Display module 418 can then drive a user interface (e.g., a displayscreen) to convey the correlation in an easy-to-understand format to theuser. For instance, augmented reality module 416 can create anavigational layer that unblocks a far right lane and blocks all otherlanes, and send this navigational layer to display module 418. Displaymodule 418 can then output a live stream of the video footage along withthe navigational layer superimposed over the video. In some embodiments,augmented reality module 416 relays the captured images to displaymodule 418, as well as determines the navigational layer and sends thenavigational layer to display module 418. Correlating the routeinformation with the identified objects in real time allows augmentedreality module 416 to provide information to display module 418 toclearly convey in an easily understood manner where the user should bewhile traveling along the route. To this end, the user interface candisplay a navigational layer that is designed to provide clear guidanceon where the user should be along the route.

An example is provided for ease of understanding. In this example,navigation module 402 conveys to augmented reality module 416 theidentity of the road on which the vehicle is currently traveling andthat a right turn is coming up in 500 feet. Navigation module 402 canalso convey which lane should be the designated lane based on theupcoming turn, such as the far right lane. Image analysis module 410conveys to augmented reality module 416 that there are four availablelanes on which the vehicle can travel, the location of each lane dividerthat divides the road into the four lanes, and conveys the identity andlocation of the available lane that is currently being traveled on bythe vehicle. Knowing that the far right lane is the designated lane perthe route information from navigation module 402, augmented realitymodule 416 can determine which of the available lanes identified byimage analysis module 410 is the far right lane and subsequently createand convey a navigational layer to display module 418 that blocks outthe three left lanes and unblocks the right-most lane. The border ofblocked and unblocked regions can be defined by the location of the lanedividers identified by image analysis module 410, which is conveyed toaugmented reality module 416. Augmented reality module 416 can alsorelay the captured images from image analysis module 410 to displaymodule 418 so that display module 418 can display raw footage of thecaptured images. Or, in some instances, display module 418 can directlyreceive captured images without being relayed by augmented realitymodule 416, as shown in FIG. 4. Display module 418 can then display thecaptured images and superimpose the navigational layer over thedisplayed captured images.

IV. User Interface for Augmented Reality

A user interface is the means by which a user and a computer systeminteract. According to some embodiments of the present disclosure, auser interface for an augmented reality device can be a display thatprovides for a visual representation of data, and in some cases, can bea touch sensitive display that can also receive inputs. This userinterface can output a series of captured images (e.g., a live stream ofvideo footage) of an area in front of a vehicle while simultaneouslysuperimposing a navigational layer over the series of captured images toclearly convey to the user where he or she should be while travelingalong a route.

A. Blocking Regions of Unintended Travel and Unblocking Regions ofIntended Travel

According to some embodiments, a navigational layer can include blockedregions and unblocked regions disposed below a horizon for clearlydisplaying where the user should be along the route. The navigationallayer can change in real time and provide information pertinent to thatspecific location along the route. For instance, the unblocked regioncan convey to the user that he or she should proceed forward, switchlanes, or turn right or left, as the user is traveling along the route.

FIG. 5A is a simplified top-down diagram of a vehicle 502 travelingalong a segment 500 of an exemplary route. An augmented reality devicecan be utilized by a user in vehicle 502, e.g., by being mounted on awindshield or mounted to the dashboard to capture images in front ofvehicle 502, as discussed herein with respect to FIG. 3. As shown inFIG. 5A, segment 500 has four available lanes 504 a-d, all of which aresubstantially straight and can be identified by an image analysismodule, e.g., image analysis module 410 discussed herein with respect toFIG. 4. Available lanes 504 a-d can be lanes in which vehicle 502 cantravel.

In some instances, only one of lanes 504 a-d may be the designated lane,e.g., a preferred lane or the lane that allows the vehicle to progressalong the route to the destination location. FIG. 5B illustrates anavigational layer superimposed over captured image 506 where only onelane, e.g., 504 c, in a multi-lane segment of road is a designated lane,according to some embodiments of the present disclosure. This may occurwhen a turn is forthcoming but still far enough away that vehicle 502can travel faster in the lane adjacent to the turn lane but can stilleasily switch into the turn lane when the turn approaches. In this case,the navigational layer can include a blocked region 508 and an unblockedregion 510 where unblocked region 510 only exposes lane 504 c. Asfurther shown in FIG. 5B, blocked region 508 and unblocked region 510can be divided by a border 512. In instances where only one lane is thedesignated lane, border 512 can extend along the lane dividers for thedesignated lane, as shown in FIG. 5B. However, if more than one lane aredesignated lanes, then the border between unblocked and blocked regionscan cross multiple lanes, as discussed herein with respect to FIG. 5C.

FIG. 5C illustrates a navigational layer superimposed over capturedimage 517 where more than one lane, e.g., 504 b-d, in a multi-lanesegment of road is a designated lane, according to some embodiments ofthe present disclosure. This may occur when vehicle 502 is travelingalong a highway and there are no immediate upcoming turns. In this case,the navigational layer can include a blocked region 518 and an unblockedregion 520 where unblocked region 520 exposes at least a portion oflanes 504 b-d. Blocked region 518 and unblocked region 520 can bedivided by a border 522, which can extend across the lane dividers ofthe designated lanes 504 b-d, as shown in FIG. 5C. It is to beappreciated that captured image 506 and 517 are each a single stillimage from respective series of captured images that, when each seriesis viewed together, forms a video, such as video footage of an area infront of vehicle 502.

According to some embodiments, the navigational layer can also includeadditional features such as one or more indicators and texts for guidingthe user forward. For instance, the navigational layer of FIG. 5B caninclude an indicator 516, text 514, and additional road information suchas speed limit information 530. Similarly, the navigational layer ofFIG. 5C can include an indicator 526, text 524, and speed limitinformation 540; and in some instances, it can also include a pop-upwindow 528. All of these features will be discussed in detail furtherherein in subsections B and C of section IV.

A typical route, however, includes many turns and is not straight theentire time. Thus, as vehicle 502 travels along the route, vehicle 502may need to change lanes to move into a better position to make theupcoming turn. FIG. 6A is a simplified top-down diagram of vehicle 602traveling along segment 600 that is a continuation of the exemplaryroute discussed in FIG. 5A, but is now approaching an upcoming rightturn. Thus, vehicle 602 can correspond with vehicle 502, and lanes 602a-d can correspond with lanes 502 a-d in FIG. 5A. As shown in FIG. 6A,the designated lane has switched from 604 c to 604 d, i.e., theright-most lane. Because the designated lane is now lane 604 d and yetvehicle 602 is still in lane 604 c, vehicle 602 needs to merge to lane604 d. This change in designated lane is shown by dotted arrow 605,which represents the old designated lane, and by solid arrow 607, whichrepresents the new designated lane.

According to some embodiments of the present disclosure, thenavigational layer can guide the user to navigate vehicle 602 into lane604 d by changing the blocked and unblocked regions. FIG. 6B illustratesa navigational layer superimposed over captured image 606 for anexemplary case where the designated lane has shifted to an adjacentlane, according to some embodiments of the present disclosure. Forinstance, the designated lane can shift from lane 604 c to adjacent lane604 d. In this case, the navigational layer can include a blocked region608 and an unblocked region 610 where unblocked region 610 includes atransition region 611 that extends from lane 604 c and gradually movesacross into lane 604 d, as shown in FIG. 6B. Transition region 611 canhave a shape that is similar to how a vehicle typically merges into anadjacent lane. If vehicle 602 travels with the shape of unblocked region610, then vehicle 602 will end up in lane 604 d and have an unblockedregion similar to unblocked region 510 discussed in FIG. 5B. However, ifvehicle 602 does not travel with the shape of unblocked region 610, thenvehicle 602 may begin to travel on blocked region 608, as shown in FIG.6C.

FIG. 6C illustrates a navigational layer superimposed over capturedimage 616 for an exemplary case where vehicle 602 is traveling in anavailable lane and not in a designated lane, according to someembodiments of the present disclosure. This may occur when vehicle 602did not follow transition region 611 to merge into the designated lane,e.g., lane 604 d, and is still in lane 604 c. In this case, thenavigational layer can include a blocked region 618 and an unblockedregion 620 where blocked region 618 is shown to appear as though vehicle602 is traveling on blocked region 618 and unblocked region 620 exposesthe adjacent lane. Having the appearance of traveling on blocked region618 while having the adjacent lane be part of unblocked region 620 cangive the user a sense that something is awry and thus create a desire tomove into unblocked region 620.

According to some embodiments of the present disclosure, thenavigational layer can also include one or more animated indicators 626to communicate a greater sense of urgency, especially when an upcomingturn is encountered. Animated indicators 626 can be a single arrow thatblinks at a certain frequency, or multiple arrows that blink in ananimated fashion, such as in a sequential order, as will be discussedfurther herein in subsection B of section IV.

After switching into the turn lane, vehicle 602 will then encounter aturn in the route. According to some embodiments of the presentdisclosure, the navigational layer can guide the user through the turn.FIG. 7A is a simplified top-down diagram of vehicle 702 traveling alongsegment 700 that is a continuation of the exemplary route discussed inFIG. 6A, but is now encountering the upcoming right turn. Thus, vehicle702 can correspond with vehicle 502 and vehicle 602, and lanes 702 a-dcan correspond with lanes 502 a-d in FIG. 5A and 602 a-d in FIG. 6A.Accordingly, as shown in FIG. 7A, the designated lane, e.g., lane 704 d,now has a curve to the right that vehicle 702 will traverse.

According to some embodiments of the present disclosure, thenavigational layer can guide the user to navigate vehicle 702 throughthe turn in lane 704 d by changing the blocked and unblocked regions.FIG. 7B illustrates a navigational layer superimposed over capturedimage 706 for an exemplary case where segment 700 of the designated laneis a turn, according to some embodiments of the present disclosure. Theturn can be an exit of a highway, an intersection between two roads, orany other transfer between roads that requires a left or right turn. Inthis case, the navigational layer can include a blocked region 708 andan unblocked region 710 where unblocked region 710 follows the shape ofthe turn, as shown in FIG. 6B. In some embodiments, blocked region 708can follow an edge of unblocked region 710 so that blocked regioncontinuously extends across an entire display screen of the augmentedreality device from its left to right edge. By extending across theentire display screen, blocked region 708 can appear as a barrier tocommunicate to the user that the route turns and that he or she shouldnot travel forward. If vehicle 702 travels with the shape of unblockedregion 710, then vehicle 702 will successfully traverse the turn andproceed along the route.

According to some embodiments of the present disclosure, thenavigational layer can also include additional features such as one ormore indicators and texts for guiding the user through a turn. Forinstance, the navigational layer of FIG. 7B can include an indicator 716that conveys a turn, text 714 that states the identity of the road thatvehicle 702 is turning on, and additional road information such as speedlimit information 730. All of these features will be discussed in detailfurther herein in subsections B of section IV.

As can be appreciated by the disclosures and illustrations of FIGS.5A-7B, providing an unblocked region that shows which lane is thedesignated lane as the user perceives it from his or her viewpointclearly conveys navigational and lane guidance information to the userin an easily understood manner. Accordingly, the augmented realitydevice according to some embodiments of the present disclosure improvesthe way in which a device can provide navigation and lane guidanceinstructions to a user.

B. Turn Indicators and Text

As mentioned herein, the navigational layer can include features such asone or more indicators and texts to aid in navigation and lane guidancealong a route, as mentioned herein. An indicator can communicate to theuser information in a visual manner. For instance, an indicator cancommunicate the current and/or upcoming direction of travel. Theindicator can be in the form of a symbol, such as a vertical arrow,slanted arrow, curved arrow, and the like, depending on whether theroute requires the vehicle to stay in the current lane, switch lanes, ormake a turn.

As an example, indicator 516 in FIG. 5B communicates to the user that heor she should continue traveling forward. This may be suitable when thevehicle is currently traveling on the designated lane and there are noimmediate upcoming turns. In another example, indicator 716 in FIG. 7Bcommunicates to the user that he or she is or will soon be turning in acertain direction. Indicator 716 can have a curved shape that followsthe direction of the upcoming turn. In some embodiments, indicator 716appears when the vehicle is a threshold distance away from the actualturn. For instance, indicator 716 can appear when the vehicle ispositioned between 100 and 500 feet away from the turn. That way, theuser can be aware of the turn before it is encountered.

In yet another example, indicator 626 in FIG. 6C can communicate to theuser that he or she should switch lanes into a designated lane. This mayoccur when the user is approaching a turn but has not followed theunblocked region and is currently in the blocked region, as discussedherein with respect to FIG. 6C. In some embodiments, indicator 626 is aslanted arrow that points toward the designated lane. Indicator 626 canbe a solid arrow or an animated indicator that blinks with a certainfrequency. The frequency at which indicator 626 blinks can be inverselyproportional to the distance between the vehicle and the turn. Forexample, the blinking frequency can increase as the vehicle gets closerto the turn. Furthermore, as shown in FIG. 6C, indicator 626 can includemore than one arrow. The arrows can blink in an animated fashion, suchas in a sequential order, or the group of arrows can all blink together.

In some embodiments, the indicator can be positioned near the center ofthe display and below the horizon, particularly in the lower half ofdisplay screen, so that it is easily viewable by the user and is notblocking the upcoming route. As an example, the indicator can bepositioned on a portion of the unblocked region as shown in FIGS. 2, 5B,5C, 6C, and 7B.

The navigational layer can also include text, such as the name of thestreet, road, or highway along which the user is currently traveling. Asan example, text 514 in FIG. 5B states the name of the road on which thevehicle is currently traveling. In another example, text 714 in FIG. 7Bstates the name of the road that the vehicle will be traveling on afterthe turn is made. In other examples, the text can indicate a distance tothe upcoming turn, as shown herein with respect to FIGS. 8B and 8C. Byproviding text to the user, the user can be more aware of his or hercurrent position along the route and of his or her next position alongthe route.

In some embodiments, such text can be positioned on the display screenin a location that is easily perceived by the user. For instance, textcan be positioned below the indicator as shown in FIGS. 2, 5B, 5C, 6C,and 7B. Although disclosures herein show the text as being positionedbelow the indicator, embodiments are not so limited. Additional oralternative embodiments can have the text positioned in any otherposition around the indicator, such as beside or above the indicator.

The navigational layer, in some embodiments, can include additional roadinformation that a user might need while traveling along the route. Forexample, the navigational layer can include speed limit information,such as speed limit information 220 shown in FIG. 2. This informationcan be positioned proximate to an edge of the display screen to avoiddistracting the user's attention away from the blocked and unblockedregions.

C. Display Area with Additional Information

According to some embodiments of the present disclosure, thenavigational layer can include a display area that provides additionalinformation to the user as he or she travels along the route. Thedisplay area can show additional information when an upcoming turn isunusual or complex and when more information about the route from adifferent perspective would be useful to the user. For instance, thedisplay area can show a top-down bird's eye view of the upcoming turnthat displays a complete profile of the turn. In some embodiments, thedisplay area is a pop-window that is displayed on a portion of thedisplay screen and that only appears temporarily. As an example, thepop-up window can appear when the vehicle is a certain distance awayfrom the unusual turn, and disappear when the vehicle has completelytraversed the turn. An exemplary navigational layer including a displayarea for showing additional information is discussed herein with respectto FIGS. 8A-8C.

FIG. 8A is a simplified top-down diagram of vehicle 802 traveling alongsegment 800 of an exemplary route, according to some embodiments of thepresent disclosure. Segment 800 of designated lane 801 is a u-turn, andvehicle 802 is shown at three points along the u-turn: point A, point B,and point C. Point A may be the point where vehicle 802 is approachingthe u-turn and is a threshold distance away from the u-turn. Thethreshold distance may be the distance at which a pop-up window appears.For instance, the threshold distance may be between 400 to 500 feet.Point B may be the point where vehicle 802 encounters the u-turn.

And, point C may be the point where vehicle 802 has completely traversedthe u-turn. In some embodiments, the pop-up window may disappear atpoint C because the user has successfully traversed the u-turn and nolonger needs information about the turn.

FIG. 8B illustrates an exemplary navigation layer including a displayarea 804 for displaying additional information after a thresholddistance has been crossed, e.g., when vehicle 802 is at point A,according to some embodiments of the present disclosure. Display area804 can be a top-down view of a surrounding area that illustrates theupcoming route. In some embodiments, the upcoming route can be ahighlighted route 808 and the vehicle can be represented as an arrowhead 806. The information shown in display area 804 can show the userwhat the turn looks like, so the user can be aware that the turn is nota typical 90-degree turn despite what indicator 810 shows.

FIG. 8C illustrates the exemplary navigation layer including displayarea 804 as vehicle 802 encounters the u-turn, e.g., when vehicle 802 isat point B, according to some embodiments of the present disclosure. Asshown in FIG. 8C, arrow head 806 can be shown as entering the u-turn;thus, the user can be aware that he or she is about to traverse theturn. In some embodiments, the navigational layer can simultaneouslyprovide blocked and unblocked regions over captured images representingthe view perceived from the user's viewpoint. For instance, thenavigational layer can include blocked region 812 and unblocked region814. Unblocked region 814 can curve to the right along with the road,and blocked region 812 can extend across the entire display screen fromthe left edge to the right edge to indicate that the user should nottravel forward.

Although FIGS. 8A-8B illustrate embodiments where a display area fordisplaying additional information is shown for traversing unusual orcomplex segments in a route, embodiments are not limited to suchsituations. In some embodiments, a display area for displayingadditional information can be utilized when a vehicle is several lanesaway from the designated lane. This may occur when the user has turnedonto a road but needs to make another turn that can only be made from alane on the opposite side of the road. In this case, it would be usefulto convey to the user that he or she needs to make a drastic lane changeto stay within the route.

By providing display area 804 in addition to blocked and unblockedregions 812 and 814, respectively, the navigational layer can clearlyconvey in an easily understood manner where the user should be whiletraveling along the route.

V. Method of Performing Lane Guidance with an Augmented Reality Device

FIG. 9 is a block diagram of a method 900 for performing navigation andlane guidance, according to some embodiments of the present disclosure.Method 900 can be performed by one or more processors that is executingcode stored in memory of an augmented reality device.

At block 902, route information can be received. In some embodiments,the route information is provided through a wireless network, e.g.,wireless network 312 in FIG. 3, by a route server, e.g., route server310 in FIG. 3, that utilizes one or more databases to provide routeinformation regarding the route of travel between an origin location toa destination location. A navigation module, such as navigation module402 of augmented reality system 400 discussed herein with respect toFIG. 4, can receive the route information from the route server. Theroute information can include data such as, but is not limited to, roadinformation (e.g., identification of roads along the route, whatsegments of those road will be traveled, the geographical position ofthose roads, the geographical position of lanes within each road, and inwhat order will the roads be traveled), turn information (e.g., whatturns are needed along the route and in what order), lane information(e.g., number and position of lanes for each road segment), and distanceinformation (e.g., distance to travel on each segment of road).

At block 904, one or more captured images can be received. An imageanalysis module, such as image analysis module 410 of augmented realitysystem 400 discussed herein with respect to FIG. 4, can receive the oneor more captured images from an image capturing device. The one or morecaptured images can be a series of captured images that together form avideo. The video can be real-time video footage of an area in front of avehicle as it traverses a route, as discussed herein with respect toFIG. 3. The video can be captured by a camera that is either built in aspart of the augmented reality device or separate from the augmentedreality device.

At block 906, the captured images can be analyzed. In some embodiments,the image analysis module can analyze the captured images to identifyone or more available lanes on the road currently traveled on by thevehicle. An available lane can be a lane in which the vehicle travel.For instance, in a four-lane highway, the available lanes can be allfour lanes. Any suitable image processing method can be used todistinguish individual lanes from the captured images, e.g., thesoftware can be trained to identify solid or dotted lines between thelanes so as to determine available lanes in the direction of travel. Asan example, the software can perform image processing to identify lanemarkers and, over time, be trained to recognize different types of lanemarkers that are occasionally used on the road. In some instances, imageanalysis module can identify the number of available lanes as well asthe position of those lanes with respect to the vehicle through theimage processing. In some additional embodiments, the image analysismodule can also analyze the captured images to identify a horizon. Theimage analysis module can identify the horizon as the border between aregion of the image that has the color of the sky and a region of theimage that has the color of the ground, e.g., asphalt, concrete, dirt,and the like. In other instances, the horizon can be identified as theborder between an area of the image that is substantially stationary andan area of the image that is constantly changing. This is because as thevehicle is traveling, regions below the horizon are constantly movingtoward the vehicle, while the sky remains relatively constant andunchanged.

At block 908, method 900 can proceed to identify one or more designatedlanes from the one or more available lanes previously identified inmethod 900. In some embodiments, an augmented reality module, such asaugmented reality module 416 of augmented reality system 400 in FIG. 4,receives the number and position of the available lane(s) with respectto the vehicle, as well as the route information. With this data, theaugmented reality module can correlate the available lane informationwith the route information to identify one or more designated lanes fromthe one or more available lanes. A designated lane can be the lane inwhich the vehicle should travel to proceed along the route.

At block 910, the captured images can be displayed along with anavigational layer superimposed over the captured images. Thenavigational layer can be any of the navigational layers discussedherein with respect to FIGS. 5B, 5C, 6B, 6C, 7B, 8B, and 8C that clearlycommunicates to a user how the user should proceed along the route. Insome embodiments, the captured images together form a real-time video ofthe area in front of the vehicle and the navigational layer issuperimposed to correlate with the video as discussed herein withrespect to the aforementioned figures. Superimposing a navigationallayer over video footage of what the user can see helps clearlycommunicate navigational and lane guidance information to the user.Thus, the augmented reality device can provide significant improvementsin navigation and lane guidance over conventional devices that merelyprovide symbols to guide a user along a route.

VI. Example Device

FIG. 10 is a block diagram of an example device 1000, which may be amobile device. Device 1000 generally includes computer-readable medium1002, a processing system 1004, an Input/Output (I/O) subsystem 1006,wireless circuitry 1008, and audio circuitry 1010 including speaker 1050and microphone 1052. These components may be coupled by one or morecommunication buses or signal lines 1003. Device 1000 can be anyportable electronic device, including a handheld computer, a tabletcomputer, a mobile phone, laptop computer, tablet device, media player,personal digital assistant (PDA), a key fob, a car key, an access card,a multi-function device, a mobile phone, a portable gaming device, a cardisplay unit, or the like, including a combination of two or more ofthese items.

It should be apparent that the architecture shown in FIG. 10 is only oneexample of an architecture for device 1000, and that device 1000 canhave more or fewer components than shown, or a different configurationof components. The various components shown in FIG. 10 can beimplemented in hardware, software, or a combination of both hardware andsoftware, including one or more signal processing and/or applicationspecific integrated circuits.

Wireless circuitry 1008 is used to send and receive information over awireless link or network to one or more other devices' conventionalcircuitry such as an antenna system, an RF transceiver, one or moreamplifiers, a tuner, one or more oscillators, a digital signalprocessor, a CODEC chipset, memory, etc. Wireless circuitry 1008 can usevarious protocols.

Wireless circuitry 1008 is coupled to processing system 1004 viaperipherals interface 1016. Interface 1016 can include conventionalcomponents for establishing and maintaining communication betweenperipherals and processing system 1004. Voice and data informationreceived by wireless circuitry 1008 (e.g., in speech recognition orvoice command applications) is sent to one or more processors 1018 viaperipherals interface 1016. One or more processors 1018 are configurableto process various data formats for one or more application programs1034 stored on medium 1002.

Peripherals interface 1016 couple the input and output peripherals ofthe device to processor 1018 and computer-readable medium 1002. One ormore processors 1018 communicate with computer-readable medium 1002 viaa controller 1020. Computer-readable medium 1002 can be any device ormedium that can store code and/or data for use by one or more processors1018. Medium 1002 can include a memory hierarchy, including cache, mainmemory and secondary memory.

Device 1000 also includes a power system 1042 for powering the varioushardware components. Power system 1042 can include a power managementsystem, one or more power sources (e.g., battery, alternating current(AC)), a recharging system, a power failure detection circuit, a powerconverter or inverter, a power status indicator (e.g., a light emittingdiode (LED)) and any other components typically associated with thegeneration, management and distribution of power in mobile devices.

In some embodiments, device 1000 includes a camera 1044. Camera 1044 canbe configured to capture images of an area in front of a vehicle, asdiscussed herein. In some embodiments, device 1000 includes sensors1046. Sensors can include accelerofeet, compass, gyrometer, pressuresensors, audio sensors, light sensors, barofeet, and the like. Sensors1046 can be used to sense location aspects, such as auditory or lightsignatures of a location.

In some embodiments, device 1000 can include a GPS receiver, sometimesreferred to as a GPS unit 1048. A mobile device can use a satellitenavigation system, such as the Global Positioning System (GPS), toobtain position information, timing information, altitude, or othernavigation information. During operation, the GPS unit can receivesignals from GPS satellites orbiting the Earth. The GPS unit analyzesthe signals to make a transit time and distance estimation. The GPS unitcan determine the current position (current location) of the mobiledevice. Based on these estimations, the mobile device can determine alocation fix, altitude, and/or current speed. A location fix can begeographical coordinates such as latitudinal and longitudinalinformation.

One or more processors 1018 run various software components stored inmedium 1002 to perform various functions for device 1000. In someembodiments, the software components include an operating system 1022, acommunication module (or set of instructions) 1024, a navigation module(or set of instructions) 1026, an image capture module (or set ofinstructions) 1028, an augmented reality module (or set of instructions)1030, and other applications (or set of instructions) 1034, such as acar locator app and a navigation app.

Operating system 1022 can be any suitable operating system, includingiOS, Mac OS, Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embeddedoperating system such as VxWorks. The operating system can includevarious procedures, sets of instructions, software components and/ordrivers for controlling and managing general system tasks (e.g., memorymanagement, storage device control, power management, etc.) andfacilitates communication between various hardware and softwarecomponents.

Communication module 1024 facilitates communication with other devicesover one or more external ports 1036 or via wireless circuitry 1008 andincludes various software components for handling data received fromwireless circuitry 1008 and/or external port 1036. External port 1036(e.g., USB, FireWire, Lightning connector, 60-pin connector, etc.) isadapted for coupling directly to other devices or indirectly over anetwork (e.g., the Internet, wireless LAN, etc.).

Navigation module 1026, image analysis module 1028, and augmentedreality module 1030 can include various sub-modules or systems, e.g., asdescribed herein with respect to FIG. 4.

The one or more applications 1034 on the mobile device can include anyapplications installed on the device 1000, including without limitation,a browser, address book, contact list, email, instant messaging, wordprocessing, keyboard emulation, widgets, JAVA-enabled applications,encryption, digital rights management, voice recognition, voicereplication, a music player (which plays back recorded music stored inone or more files, such as MP3 or AAC files), etc.

There may be other modules or sets of instructions (not shown), such asa graphics module, a time module, etc. For example, the graphics modulecan include various conventional software components for rendering,animating and displaying graphical objects (including without limitationtext, web pages, icons, digital images, animations and the like) on adisplay surface. In another example, a timer module can be a softwaretimer. The timer module can also be implemented in hardware. The timemodule can maintain various timers for any number of events.

The I/O subsystem 1006 can be coupled to a display system (not shown),which can be a touch-sensitive display. The display displays visualoutput to the user in a GUI. The visual output can include text,graphics, video, and any combination thereof. Some or all of the visualoutput can correspond to user-interface objects. A display can use LED(light emitting diode),

LCD (liquid crystal display) technology, or LPD (light emitting polymerdisplay) technology, although other display technologies can be used inother embodiments.

In some embodiments, I/O subsystem 1006 can include a display and userinput devices such as a keyboard, mouse, and/or track pad. In someembodiments, I/O subsystem 1006 can include a touch-sensitive display. Atouch-sensitive display can also accept input from the user based onhaptic and/or tactile contact. In some embodiments, a touch-sensitivedisplay forms a touch-sensitive surface that accepts user input. Thetouch-sensitive display/surface (along with any associated modulesand/or sets of instructions in medium 1002) detects contact (and anymovement or release of the contact) on the touch-sensitive display andconverts the detected contact into interaction with user-interfaceobjects, such as one or more soft keys, that are displayed on the touchscreen when the contact occurs. In some embodiments, a point of contactbetween the touch-sensitive display and the user corresponds to one ormore digits of the user. The user can make contact with thetouch-sensitive display using any suitable object or appendage, such asa stylus, pen, finger, and so forth. A touch-sensitive display surfacecan detect contact and any movement or release thereof using anysuitable touch sensitivity technologies, including capacitive,resistive, infrared, and surface acoustic wave technologies, as well asother proximity sensor arrays or other elements for determining one ormore points of contact with the touch-sensitive display.

Further, the I/O subsystem can be coupled to one or more other physicalcontrol devices (not shown), such as pushbuttons, keys, switches, rockerbuttons, dials, slider switches, sticks, LEDs, etc., for controlling orperforming various functions, such as power control, speaker volumecontrol, ring tone loudness, keyboard input, scrolling, hold, menu,screen lock, clearing and ending communications and the like. In someembodiments, in addition to the touch screen, device 1000 can include atouchpad (not shown) for activating or deactivating particularfunctions. In some embodiments, the touchpad is a touch-sensitive areaof the device that, unlike the touch screen, does not display visualoutput. The touchpad can be a touch-sensitive surface that is separatefrom the touch-sensitive display or an extension of the touch-sensitivesurface formed by the touch-sensitive display.

In some embodiments, some or all of the operations described herein canbe performed using an application executing on the user's device.Circuits, logic modules, processors, and/or other components may beconfigured to perform various operations described herein. Those skilledin the art will appreciate that, depending on implementation, suchconfiguration can be accomplished through design, setup,interconnection, and/or programming of the particular components andthat, again depending on implementation, a configured component might ormight not be reconfigurable for a different operation. For example, aprogrammable processor can be configured by providing suitableexecutable code; a dedicated logic circuit can be configured by suitablyconnecting logic gates and other circuit elements; and so on.

Any of the software components or functions described in thisapplication may be implemented as software code to be executed by aprocessor using any suitable computer language such as, for example,Java, C, C++, C#, Objective-C, Swift, or scripting language such as Perlor Python using, for example, conventional or object-orientedtechniques. The software code may be stored as a series of instructionsor commands on a computer readable medium for storage and/ortransmission. A suitable non-transitory computer readable medium caninclude random access memory (RAM), a read only memory (ROM), a magneticmedium such as a hard-drive or a floppy disk, or an optical medium suchas a compact disk (CD) or DVD (digital versatile disk), flash memory,and the like. The computer readable medium may be any combination ofsuch storage or transmission devices.

Computer programs incorporating various features of the presentinvention may be encoded on various computer readable storage media;suitable media include magnetic disk or tape, optical storage media suchas compact disk (CD) or DVD (digital versatile disk), flash memory, andthe like. Computer readable storage media encoded with the program codemay be packaged with a compatible device or provided separately fromother devices. In addition program code may be encoded and transmittedvia wired optical, and/or wireless networks conforming to a variety ofprotocols, including the Internet, thereby allowing distribution, e.g.,via Internet download. Any such computer readable medium may reside onor within a single computer product (e.g. a solid state drive, a harddrive, a CD, or an entire computer system), and may be present on orwithin different computer products within a system or network. Acomputer system may include a monitor, printer, or other suitabledisplay for providing any of the results mentioned herein to a user.

Although the invention has been described with respect to specificembodiments, it will be appreciated that the invention is intended tocover all modifications and equivalents within the scope of thefollowing claims.

1. (canceled)
 2. A computer-implemented method for guiding a user alonga route between at least two locations, the method comprising, at amobile device that has a display and a camera positioned to take imagesin front of a vehicle: storing route information, the route informationcomprising geographical positions of roads and designated lanes oftravel for each of a plurality of segments along the route; receiving aseries of images captured by the camera along the route; analyzing theseries of images to determine one or more available lanes of travelalong the route; identifying one or more of the designated lanes fromthe one or more available lanes based on the analyzing of the series ofimages and the route information; and displaying the series of imagesalong with a navigational layer superimposed over the series of imagesto the user, the navigational layer indicating the one or moredesignated lanes.
 3. The method of claim 2, wherein the navigationallayer comprises an unblocked region revealing the one or more designatedlanes.
 4. The method of claim 2, wherein the navigational layer furthercomprises one or more guidance arrows positioned near a center of atleast one image of the series of images.
 5. The method of claim 4,wherein the one or more guidance arrows are positioned point toward theone or more designated lanes when the user is positioned outside of theone or more designated lanes along the route.
 6. The method of claim 4,wherein the one or more guidance arrows are positioned over an unblockedregion.
 7. The method of claim 6, wherein the one or more guidancearrows indicate an upcoming turn or that the user should switch lanes.8. The method of claim 7, wherein the one or more guidance arrows areanimated such that the one or more guidance arrows blink at a frequencyinversely proportional to a distance to the upcoming turn.
 9. The methodof claim 8, wherein the one or more guidance arrows comprise multiplearrows that blink in a sequential order.
 10. The method of claim 2,wherein the navigational layer comprises a blocked region covering otherlanes captured in the series of images that are outside of the one ormore designated lanes.
 11. The method of claim 2, wherein thenavigational layer further comprises a pop-up window that temporarilyappears as the user approaches a turn.
 12. The method of claim 11wherein the pop-up window appears within a threshold distance to theturn and disappears after the turn has been traversed by the user. 13.The method of claim 11 wherein the pop-up window displays a top-downview of an area surrounding the user that includes a complete profile ofthe turn.
 14. A computer product comprising a non-transitory computerreadable medium storing a plurality of instructions that when executedcontrol a mobile device including a display, a camera positioned to takeimages in front of a vehicle, and one or more processors for guiding auser along a route between at least two locations, the instructionscomprising: storing route information, the route information comprisinggeographical positions of roads and designated lanes of travel for eachof a plurality of segments along the route; receiving a series of imagescaptured by the camera along the route; analyzing the series of imagesto determine one or more available lanes of travel along the route;identifying one or more of the designated lanes from the one or moreavailable lanes based on the analyzing of the series of images and theroute information; and displaying the series of images along with anavigational layer superimposed over the series of images to the user,the navigational layer indicating the one or more designated lanes. 15.The computer product of claim 14, wherein the navigational layercomprises an unblocked region revealing the one or more designatedlanes.
 16. The computer product of claim 14, wherein the navigationallayer further comprises one or more guidance arrows positioned near acenter at least one image of the series of images.
 17. The computerproduct of claim 16, wherein the one or more guidance arrows pointtoward the one or more designated lanes in an event the user ispositioned outside of the one or more designated lanes along the route.18. A mobile device for guiding a user along a route between at leasttwo locations, the mobile device comprising: a camera positioned to takeimages in front of a vehicle; a display; and one or more processorscoupled to the camera and the display, the one or more processorsconfigured to: receive route information from a route server, the routeinformation comprising geographical positions of roads and designatedlanes of travel for each of a plurality of segments along the route;receive a series of images captured by the camera along the route;analyze the series of images to determine one or more available lanes oftravel along the route; identify one or more of the designated lanesfrom the one or more available lanes based on the analyzing of theseries of images and the route information; and display the series ofimages along with a navigational layer superimposed over the series ofimages to the user, the navigational layer indicating the one or moredesignated lanes.
 19. The mobile device of claim 18, wherein thenavigational layer comprises an unblocked region revealing the one ormore designated lanes.
 20. The mobile device of claim 18, wherein thenavigational layer further comprises one or more guidance arrowspositioned near a center of at least one image of the series of images.21. The mobile device of claim 20, wherein the one or more guidancearrows are positioned point toward the one or more designated lanes inan event the user is positioned outside of the one or more designatedlanes along the route.